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Abstract 

Interactive behavior among humans is governed by the dynamics of movement synchronization in a variety of repetitive 
tasks. This requires the interaction partners to perform for example rhythmic limb swinging or even goal-directed arm 
movements. Inspired by that essential feature of human interaction, we present a novel concept and design methodology 
to synthesize goal-directed synchronization behavior for robotic agents in repetitive joint action tasks. The agents' tasks are 
described by closed movement trajectories and interpreted as limit cycles, for which instantaneous phase variables are 
derived based on oscillator theory. Events segmenting the trajectories into multiple primitives are introduced as anchoring 
points for enhanced synchronization modes. Utilizing both continuous phases and discrete events in a unifying view, we 
design a continuous dynamical process synchronizing the derived modes. Inverse to the derivation of phases, we also 
address the generation of goal-directed movements from the behavioral dynamics. The developed concept is implemented 
to an anthropomorphic robot. For evaluation of the concept an experiment is designed and conducted in which the robot 
performs a prototypical pick-and-place task jointly with human partners. The effectiveness of the designed behavior is 
successfully evidenced by objective measures of phase and event synchronization. Feedback gathered from the participants 
of our exploratory study suggests a subjectively pleasant sense of interaction created by the interactive behavior. The 
results highlight potential applications of the synchronization concept both in motor coordination among robotic agents 
and in enhanced social interaction between humanoid agents and humans. 
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Introduction 

Synchronization is frequently observed across different modal- 
ities and situations. In particular, the synchronization of move- 
ments is found to play an essential role in the interactive behavior 
of humans. Due to its ubiquity in human life, interpersonal 
synchronization is experimentally investigated in various tasks that 
require jointly performed movements in a shared workspace: 
When walking in a group, humans tend to synchronize their gait 
[1]. Two people sitting next to each other in rocking chairs are 
found to synchronize their rocking movements [2], even if the 
natural frequencies of the chairs differ. Similar behavior is 
observed in laboratory tasks such as pendulum swinging [3] or 
pure leg movements [4]. Even during goal-directed tapping that 
requires precise arm movements [5], synchronization among 
human dyads is emerging naturally without being instructed or 
demanded for the task. In this task, interpersonal movement 
synchronization can be clearly quantified as a coupled dynamical 
process [6]. Studies on the social aspects of synchronization 
highlight that falling into synchrony with partners enhances 
perceptual sensitivity toward each other, fosters cooperative 
abilities [7] and leads to the attribution of more positive 
characteristics to the interaction partner [8]. These works give 



rise to the hypothesis, that bidirectional motor coordination with 
synchronization as its key concept is a promising way to increase 
the social competence of robots when interacting with humans [9] . 

Inspired by the appealing prospect to enrich the interaction 
repertoire of robots, this article addresses the challenge of 
designing interactive behavior for artificial agents engaging in 
repetitive joint action tasks. These tasks involve actions performed by 
two or more individuals in a common social setting, inducing 
action coordination in space and time [10]. Based on synchroni- 
zation theory of coupled dynamical systems [11], we present a 
synchronization concept for repetitive, goal-directed movements 
composed by mixed continuous and discrete primitives. 

Movement Synchronization among Humans and 
Machines 

One line of research on human synchronization behavior 
follows the dynamical systems approach. Patterns of coordination 
are considered to result from attractors of dynamical systems, that 
model interconnected perception-action loops [12]. This concept 
is also called behavioral dynamics [13]. Investigating intrapersonal 
limb coordination, Haken et al. [14] propose a minimal dynamical 
model of coupled oscillators, which is known as the Haken-Kelso- 
Bunz (HKB) model. It reproduces the main coordination features 
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observed during rhythmic bi-manual finger-tapping. The HKB 
model family qualitatively explains interpersonal movement 
synchronization in rhythmic paradigms as well [2,15]. In this 
vein, the rigorous design of rhythmic movement behavior by 
dynamical systems in the state space is performed by Jirsa and 
Kelso [16], which the authors call the excitator model. Common 
to these approaches is the monolithic encoding of movement 
coordination and reproduction, making them rather task specific. 
A second line of research on human synchronization behavior is 
devoted to the intended synchronization of human rhythmic 
movements with respect to purely discrete, periodic stimuli such as 
auditory metronome beats, which is often called sensorimotor 
synchronization (SMS). Linear models of asynchrony correction 
based on the Wing-Kristofferson model [17] explain perceptual 
and motor variabilities from an information-processing point of 
view, see [18,19] for an exhaustive review. It is recendy debated 
whether hybrid incarnations of both the dynamical systems 
approach and the linear error correction concept may exist [20] 
or not [21], or whether these model classes simply account for 
different synchronization processes present in the task [22]. 
Irrespective of the underlying process, it is found that discrete 
perceptual information such as distinguishable events during 
continuous movements provides anchoring points for time keeping 
with a stimulus and, thus, fosters human SMS [23]. Notably, 
humans are found to rapidly adjust their pacing toward each other 
during dyadic finger tapping, thus improving coordination by 
mutually coupled SMS [24]. 

Limit cycle systems creating rhythmic movement based on self- 
sustained oscillators are also called central pattern generators 
(CPGs) in robotics. Entrainment tasks, such as robot drumming 
[25], are modeled by CPGs, where phase locking regarding the 
beats is achieved. An extension of the CPG approach by 
reconfigurable dynamical systems is proposed by Degallier et al. 
[26] to generate mixed discrete and rhythmic movements in 
multiple degrees of freedom. The encoding of periodic movements 
based on adaptive frequency oscillators is realized in Gams et al. 
[27] and developed further by Petric et al. [28]. Frequency and 
phase tuning shows a rather slow rate of convergence for non- 
stationary trajectories. Though CPGs model robust and flexible 
motor behavior, an open issue is the missing methodology to 
systematically design and specify CPGs in a task-oriented way. For 
profound reviews on the design and application of CPGs, the 
reader is referred to [29,30]. Some works investigate human- 
machine rhythmic coordination. Mutual entrainment of move- 
ments is achieved by rendering visual or acoustic stimuli to the 
human as real-time feedback. The concept of virtual partner 
interaction (VPI) is introduced in [31]. In a proof-of-concept 
implementation, the coordination of finger movements between a 
human and a visually-rendered, virtual agent driven by the HKB 
model is explored systematically. In various applications, rhythmic 
entrainment between humans and robots is investigated. Popular 
examples are human-robot rope turning [32,33] or the imitation 
of human rhythmic movements of selected target frequencies [34] 
by means of phase-locked loops (PLL) [35] . Both human-robot 
handshaking [36] and physical assistance for rhythmic knee 
movements [37] are realized based on the Matsuoka neural 
oscillator [38]. Here, Sato et al. [39] achieve encoding of rhythmic 
movements and implicit synchronization through an on-line 
polynomial design of the attractor dynamics, which is originally 
proposed by Okada et al. [40]. 

However, the above works focus either on fundamental research 
of human synchronization behavior or, within human-machine 
interaction, on applications in purely rhythmic tasks. To the 
authors' best knowledge, none of the existing works, except our 



previous [5,6], analyzes and models synchronization of hybrid 
action tasks composed by mixed continuous and discrete 
primitives with application to human-robot dyads. 

Contribution 

In this article, we develop a concept of movement synchroni- 
zation for repetitive joint action tasks. Those tasks are assumed to 
be described by closed movement trajectories that can be goal- 
directed and comprise multiple primitives. The modeling concept 
pursued in our analytical work on human synchronization 
behavior [6] is generalized to enable the design of synchronization 
behavior for robotic agents in a wide range of tasks: Based on 
oscillator theory, limit cycle representations of the trajectories in 
state space are used to derive the phase variable, even for 
sequences of multiple primitives. Relevant synchronization modes 
within pairs of limit cycles are synthesized considering both 
continuous phases and discrete events from a unifying point of 
view. In line with the behavioral dynamics perspective [13,41], we 
design a dynamical process to synchronize the derived modes. 
Movement generation is addressed as well, in order to enable a 
robotic agent that is equipped with synchronization behavior 
engage in repetitive joint action tasks. The presented experimental 
study employing a full-sized, anthropomorphic robot serves not 
only as proof of concept; it also defines a versatile testbed for the 
investigation of human-robot interactive behavior in realistic 
settings. 

The remainder of this article is organized as follows. First, we 
clarify the required assumptions and definitions. Based on those, 
synchronization modes are analyzed and dynamical synchroniza- 
tion behavior is designed accordingly. Next, the required 
transformations between phase variables and movement trajecto- 
ries are developed. After the design concept, we describe the 
human-robot experiment, its implementation and the applied 
measures. A detailed assessment and evaluation of the implement- 
ed synchronization behavior is presented. After a discussion of the 
results and insights, we sum up and draw the conclusions. 

Bold characters are used for denoting vectors in this article. 
Superscripts 11 and b are used when variables belonging to agent 'a' 
and agent 'b' need to be distinguished. For clarity these 
superscripts are omitted otherwise. 

Problem Setting and Definitions 

This section provides the reader with the formal representations 
and definitions that are used in this article to characterize the joint 
action task as well as movement synchronization. 

Representation of Repetitive Joint Action Tasks 

The notion of joint action [10], originated from cognitive 
psychology, is adopted in this work, whereas we extend joint action 
to robotic agents as well. Let each agent's part of the joint action 
task, which we call the individual task, be represented by a state 
trajectory i.e. the evolution of the vector of relevant states 
%eR" over time t. The state vector can be composed by the 
configuration of the agent's limbs, their hand (effector) position, or 
any other coordinates that describe the movements associated with 
the individual task. 

Mote. A certain set of states is considered suitable if the 
information conveyed through the chosen description allows to 
explain and model the synchronization behavior of the agents. 

Limit cycle trajectory. The concept of movement synchro- 
nization exploits the repetitive aspect of the individual task. 
Therefore, the state trajectory is required to be cyclic, i.e. for any 
time t and finite time spans T the condition 
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«*+r>=«f) (l) 

holds. The smallest 7">0 which fulfills (1) is denoted the period. It 
follows that the state space representation of is of circular 
shape, which is denoted the limit cycle y, see Fig. 1. Due to 
interaction between the agents, the period T is time-varying and 
consequently, y is strictly speaking not periodic. 

Note. For trajectories obtained from noisy measurements, 
condition (1) is relaxed by examining the return times to the 
Poincare secant surface [42], allowing for T) x ^(t). 

Primitives, events and durations. The limit cycle y is 
assumed to be composed by a number of L segments ji/ £y in an 
ordered sequence l=\,...,L. These are called primitives. Each 
primitive Hi is delimited by two segmentation points, the start 
point = £(//_ i_) and the end point = ^(f/„), as illustrated in 
Fig. 2(a). The positive index ieN denotes the rth period. The 
period is taken by 7. = £_,„■ — toj in the following. It has to be noted 
that %l=£o an d tL,i = to,i+i respectively, since is cyclic. The 
times tt-ij and */„• are called events, see Fig. 2(b). Without loss of 
generality, we choose segmentation points featuring discriminable 
events, such as local extrema of the movement with vanishing 
velocity [43] . Segmentation points with zero or negligible velocity 
persisting for non-zero time intervals are considered as postures 
[44] and separate dwell primitives respectively. Those dwell 
primitives are also delimited by event pairs, denoting the times of 
movement stop and start. Discriminable events in cyclic trajecto- 
ries are shown to support human mechanisms of temporal error 
correction [23], and thus affect human synchronization behavior. 

Mote. The segmentation points are assumed to be such that any 
task-related constraints on the state can be satisfied, e.g. goal 
points or forbidden state regions. 

The times ti—ij and define the primitive duration 

T,j = t u -t,_ u . (2) 

Relating the current primitive duration 7/ = 7/„- and the current 
period T =T t with index i such that to,i<t<tL,i, we further 
define the relative primitive duration 

d,= ^, with X> = 1. (3) 
i=\ 

The distribution d=[d\ ... dj\ gathering d\ in a vector scales 
the primitive durations 7/ under modulations of 7. 



A 



B 



_____ Ml 


X 










Y 






M3 ... 




i T " i ^ : 


TV,i'"-- 


to,i + T 

— ^ ► 


to, 1 1 1 , 1 t2,l 
1 1 1 




t 3 ,1 t 

1 



0 I n 2n 



Figure 2. Characterization of the limit cycle. (A) Exemplary limit 
cycle y with the state ( = [x i] T and L — 3 primitives. The segmentation 
points are given by the intersection of y with the abscissa. (B) The 
corresponding events t[,\, primitive durations 7/,i and the uniformly 
growing phase 0 depicted for i'=l. 
doi:10.1371/journal.pone.0095195.g002 

cycle oscillators. The notion of phase is introduced to describe the 
motion of the state on the limit cycle. The definition of 
synchronization relates both the phase and events of a pair of 
limit cycles to each other and therefore, characterizes coordination 
in time. 

The phase variable. Through a coordinate transformation, 
the limit cycle is re-parameterized by the one-dimensional variable 
9 that is called the phase and describes the motion on y and the 
in — l)-dimensional vector of amplitudes that describe motions 
transverse to y. This transformation is not unique, and thus, 
different decompositions can be found for a certain limit cycle 
[45]. In our setting of goal-directed tasks, we assume the 
amplitudes to be constrained by the segmentation points 
delimiting the primitives, compactly gathered in 3 = [£, ... £_J. 
Consequendy, only the phase is considered to be governed by 
synchronization in the following. 

Among all possible transformations, we choose the phase 
obtained from the harmonic phase oscillator, which is one of the 
simplest oscillator models. Its unperturbed oscillations evolve at 
constant phase velocity 9 = 0}, with w denoting the natural 
frequency. Accordingly, its phase trajectory is defined 

9(t) = cot + 0 O , (4) 

which i. ^ — y » BV «» 5 - % we «- 
define the phase to be angular and 27i-periodic, evaluating 



Synchronization of Limit Cycle Pairs 

The limit cycle y is assumed to be originated from a self- 
sustained oscillation, which allows us to apply the theory of limit 




Figure 1. Limit cycle y of an exemplary cyclic state trajectory 

<_,(/) in its state space with <_,eR 3 . If y is cyclic, yet not closed exactly, 
the period T is determined by the return time of { to the Poincare 
secant surface S. 

doi:1 0.1 371 /journal.pone.00951 95.g001 



9(iT) = 2m + 9 0 , (5) 

with any initial phase 9q. Finally, the phase 9{t) needs to be 
uniquely related to the state £(/). We deliberately choose 9q such 
that 0(?o,]) = O is anchored to the point <J(?o,l) = marking the 
event ?o,i, cf. Fig. 2(b). The phase of a stationary limit cycle with 
constant period T is readily given by (4), which is analogous to the 
marker technique in [42] . The important case of a non-stationary 
limit-cycle with a-priori unknown period T(t) is addressed later. 

Note. The above transformation can be understood as a 
decomposition of the task into the phase, which is the voluntary 
degree of freedom available for synchronization, and the 
amplitudes, which are the remaining degrees of freedom necessarily 
complying with the task goals. 
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Phase and event synchronization. With the above defini- 
tion of phase and under the assumption that the phase is 
originated from a self-sustained oscillating entity, a synchroniza- 
tion problem between a pair of phase oscillators is posed 
accounting for a dyad's coordination in time. The oscillators are 
assumed to be mutually coupled through some coupling function 
and completely described by their phases 0 d ^ b (t) defined on the 
limit cycles y a / b . If the phase difference 

<D(0 = e a (0-e b (0 (6) 

is bounded by a positive constant eeM 

|(D(f)|<£ W, (7) 

the limit cycles show phase synchronization [1 1] of order 1:1. Higher 
order synchronization is not addressed in this article for the sake of 
simplicity. 

In addition, the quasi-simultaneous appearance of event pairs is 
considered, known as event synchronization [46]. Let denote 
the time of the /th event in the iih period of y. We define, that the 
event pair denoted by the tuple {i%,t b h ) shows event synchronization, if 
the events keep the temporal relation 

|^a-^ b , a+m |<A? Vf\ (8) 

with some time span At. Choosing m = const., with meN ensures 
to test for event synchronization of order 1:1. The choice of A? is 
considered as problem dependent. To avoid ambiguities, a 
reasonable upper bound is given by 

Ar " \f$ {T P^ T v+^ T l^ T l + u^> (9) 

which is half the minimum primitive duration or half the 
minimum inter-event distance in the neighborhood of the 

considered pair ^t'p,t b h ^. 

Note. The above notion of event synchronization implies phase 
synchronization, since the time lag and thus, the phase difference 
between the considered events is bounded. Event synchronization 
depending on the definition of relevant events provides a problem- 
specific characterization of the temporal organization of two limit 
cycles. 

Design of Synchronization Behavior 

Following the above definitions of phase and event synchroni- 
zation and inspired by principles of human movement synchro- 
nization, in this Section we design synchronization behavior with 
application to repetitive joint action tasks. Accounting for the 
derived descriptions of possible synchronization modes, a unified 
synchronization process is developed. 

Synchronization Modes 

After analyzing the common modes of the synchronization 
between quasi-harmonic trajectories, which usually result from 
rhythmic movement tasks, we broaden the repertoire of potential 
synchronization modes between limit cycles featuring multiple 
primitives and events. 

Modes between harmonic limit cycles. Research on 
movement synchronization within human dyads has mainly 



focused on task paradigms requiring purely rhythmic movements 
such as finger tapping, leg or pendulum swinging. These tasks are 
usually described by one-dimensional motion trajectories, e.g. with 
the state ^eR 2 embedded in a position-velocity state space. 
Typically, each period of the trajectory is composed by two nearly 
equal and sinusoidal half-periods, allowing to treat the oscillation 
as harmonic. Following the definitions made above, the limit cycle 
y of the state trajectory £(t) is segmented into L = 2 primitives 
Hi czy, /= 1,2, which are symmetric due to their relative primitive 
durations with d/= j being constant and equal, cf. Fig. 3. For pairs 
of limit cycles y' d l b originated from harmonic oscillations, the 
notions of the in-phase and the anti-phase relation usually 
characterize the common modes of synchronization. When we 
calculate the relative phase difference 

(D r (X) = (D(Y)mod27r, (10) 

with fl>(7) from (6) and mod denoting the mathematical modulo 
division, the in-phase and the anti-phase mode map to <I> r = 0 and 
<3> r = 71 respectively, cf. Fig. 3(b). 

These modes are equivalently described by event synchroniza- 
tion according to the above definition of synchronization. 
Evaluating the phase (4) at the event ?/„■ yields with (3) 

/ 

9(t u ) = 2n^dj + 2n(i-l). (11) 

7=1 

Thus, we obtain 0(fi,<) mod 2n = n and 8(t 2 ,d mod 2n = 0 for 
symmetric primitives with d^ b = d^f° = -. It follows that the 
relative phase difference (10) evaluates fl> r e = 0 and <J> re = n, if the 
event pairs (^1/2^1/2) an d ( ? i/2' ? 2/l) a PP ear synchronized. 
Summing up, quasi-harmonic cycles are considered to be 
composed by two symmetric primitives and events respectively. 
Their common synchronization modes are sufficiently described 
by the phase dynamics of coupled oscillator models, e.g. 
[6,14,47,48]. 

Modes between multiple-primitive limit cycles. In re- 
petitive joint action tasks, the limit cycles y' d ^ b represent the agents' 
individual tasks. Those can be composed by different sequences of 
multiple primitives, i.e. with the number of primitives L d l b >2, the 
distributions of relative primitive durations d d ^d b , or both. Here, 
the relevant modes of synchronization are assumed to describe the 
(simultaneous) synchronization of one or more event pairs (tp,^), 
see the modes in Fig. 4(c)— (d). 

The example in Fig. 4 illustrates, that phase synchronization is 
not sufficient to describe all of these modes. Phase synchronization 
models stable equilibrium points <J> r e of the phase difference which 
lead to O r (?)- >( I ) r,e and imply T d — T b —>0 in the domains of 
attraction. This allows to synchronize single event pairs, like the 
one depicted in Fig. 4(c). If the within-cycle distributions of events 
differ d d # d b like in our example, the simultaneous synchroniza- 
tion of not more than one event pair is explained by the phase 
dynamics, since the events scale under changes of T' d l b with the 
distributions d d l b , which are, however, left uncontrolled. Obvi- 
ously, the simultaneous synchronization of multiple event pairs 
requires an additional adjustment of see Fig. 4(d). 

Note. Only a task-dependent subset of events might be 
synchronized, e.g. only those that are perceived by the interaction 
partner. 
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Figure 3. Modes between harmonic oscillations. Phase synchronization resulting in in-phase or anti-phase relations comes about with event 
synchronization with respect to the segmentation points C\ b and . (A) Motion trajectories j?/ b (i) describing the temporal relation. (B) Their limit 
cycle representations y a / b in a position-velocity state space, illustrating the phase difference. 
doi:1 0.1 371 /journal.pone.00951 95.g003 



Dynamical Synchronization Process 

Synchronization behavior is modeled in line with the dynamical 
systems approach [13], which explains stable behavioral patterns by 
attractors of dynamical systems. First, we review the phase 
dynamics modeling the synchronization of human dyads perform- 
ing quasi-harmonic limit cycles in a goal-directed movement task. 
The above analysis shows, that phase synchronization is able to 
account only for a limited number of possible synchronization 
modes. Therefore, we design a unified synchronization process 
that features the simultaneous synchronization of multiple event 
pairs. 

Model of coupled phase oscillators. In accordance to the 
definition of phase synchronization, the model structure is given by 
a pair of cross-coupled phase oscillators 



0 a =co' l + G\e h -6 d ) (12) 



e h =o} h + G b (6 ll -e h ), (13) 

with the natural frequencies co a / b , and the coupling functions G a / b 
depending on the phase difference between the oscillators. By 
subtracting (13) from (12), we obtain the phase difference 
dynamics 




B 



4-+ 



t a 



44-^ t- 



t a 2 




4 — ? 1- 



' L 0' l 0 ; 



4 ^ 



(t a t b i 



t?,t$j 



Figure 4. Event synchronization of heterogeneous limit cycle pairs. (A) Exemplary limit cycles y a / b with L a = 2 and L b = 3 primitives in 
position-velocity state spaces. The evolution of the events in y a ' b / (B) without synchronization, (C) with synchronization of the event pair (?j5,? b ) as 
achieved by phase synchronization, (D) with additional synchronization of (t\,tj). The shaded areas denote the time span At defining event 
synchronization. 

doi:1 0.1 371 /journal.pone.00951 95.g004 



PLOS ONE | www.plosone.org 



5 



April 2014 | Volume 9 | Issue 4 | e95195 



Synchronization Behavior for Human-Robot Joint Action 



<j> = Aco + i7(€>), (14) 
with <j> = 8 — 8 and the frequency detuning 



Aco = co a -co b . (15) 

The function H is the vector field of fl> forming the attractor 
landscape, and thus, the preferred modes of phase synchronization. 

Mote. Synchronization behavior is assumed to be voluntary and 
compliant with the task-related goals. We therefore require the 
coupling functions to be weak and 27i-periodic, i.e. equilibrium 
points O e are equivalently described by equilibrium points fl> r , e of 
the relative phase difference (10) between the oscillators. Conse- 
quently, a large enough frequency detuning Aco completely 
eliminates stable attractors, which is found to be in line with 
unintentional coordination behavior of humans [49]. 

In the following, we review a realization of the coupling 
functions G a / b that accounts for the observed process of inter- 
human movement synchronization [6]. The proposed model 
structure is based on the classical Kuramoto model [50]. Its equations 
of motion read 

8* = af+Ksin[2(e h -ff i )] (16) 



A 




Figure 5. R.h.s. terms of the phase difference dynamics (14) 
over Oe[0,2#]. The intersection points of the graphs of Aco and 
— i/(cD) denote equilibria with <I> = 0. The vector fields are illustrated on 
the abscissae. (A) The extended Kuramoto model featuring two equally- 
spaced attractors. (B) Exemplary phase dynamics featuring three 
attractors determined via (19). 
doi:10.1 371/journal.pone.00951 95.g005 



0 b = co b + i:sin[2(fJ il -e b )], (17) 

which we call the extended Kuramoto model. The natural frequencies 
model the individually preferred speed of task performance, 
whereas the sinusoidal coupling with the isotropic gain K 
replicates the dyad's interactive behavior. We obtain the phase 
difference dynamics 

<J5=Affl-2Zsiri(2<J>), (18) 

featuring two point attractors around $ r ,e,l =0 and ( J ) r,e,2 = 71 , see 
Fig. 5(a) for a visualization of the vector field. The dynamics (18) 
replicate the synchronization process of human dyads that leads to 
in-phase and anti-phase modes between quasi-harmonic, yet goal- 
directed movements [5]. Details concerning this model, e.g. its 
stability properties can be found in [6] . 

Note. The extended Kuramoto model implies equal attractor 
strengths, as both attractors were met nearly equally often in the 
experimental task. 

Synchronization of single event pairs. In-phase and anti- 
phase synchronization between harmonic limit cycles is now 
generalized to synchronization modes of single event pairs in 
arbitrary combinations. Again, stable modes of synchronization 
are mapped to stable equilibrium points fl> re of the vector field H. 
The values of 0>i#, i.e. the locations in the attractor landscape, 
depend on the definition of the events for which the initial 
phases (4) evaluate 8^ b =0. It makes sense to define them such 
that the pair (/q,?o) denotes a synchronization mode, with the 
corresponding attractor cD r £ . = 0. Using (11), the synchronization 
mode of any event pair (tp,fy) is then expressed by the 
equilibrium phase difference 



P l b \ 

<b e = 8*(i%)-8 h (t%) = 2n E^'-E^ ■ ( 19 ) 



For each event pair representing a synchronized mode, the 
vector field H of the phase difference dynamics (14) needs to 
feature a point attractor O r>e , which is obtained from (19) with (10), 
see Fig. 5(b) for an example. The following points summarize the 
properties common to the design of the vector field H: 

• The phase plot is of oscillating shape, modeling an alternating 
sequence of attractors and repellors. 

• The gradient and extrema in the vicinity of an equilibrium 
point 3> r-e define its strength and region of attraction 
respectively [6], given a certain frequency detuning Aco. 

• In order to obtain relative synchronization, we require 
H{<b)=H{<b+2n). 

• In contrast to the extended Kuramoto model and similar 
coordination models, symmetry H{ — 0)= —H(<S>) is generally 
not fulfilled. 

• Positive (negative) values Aco yield positive (negative) shifts of 
the attractor points. 

Mote. The attractor landscape of the phase dynamics becomes 
time-varying, if the relative primitive durations d' l ^ b are subject to 
adjustment. 

Synchronization of multiple events pairs. The coupled 
process (12), (13) accounts for synchronization modes that can be 
achieved by mutual entrainment of both periods and phase 
difference within certain domains of attraction. However, the 
simultaneous synchronization of multiple event pairs remains 
generally unexplained, as pointed out in the previous section. 
Therefore, the relative primitive durations (/ a ' b are proposed as 
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additional degrees of freedom, governed by a cross-coupled 
dynamical process of the form 



By (22), normalization is preserved. The gain Kd>§ in (23) 
enforces the solution c/f = c/3, s to be stable, saturated by 



d = D*(d\d h ), 



(20) 



4y- 



if d a ix 
otherwise 



' b <rl a 
3 — "2,h 



d b = D b (d h ,d% (21) 

with rf a ' subject to normalization 2_^i-\ *7 = ^ ^ n ^8' 6, the 
degrees of freedom of the overall synchronization process are 
illustrated for the above mode with respect to two event pairs. 
Synchronization modes that would require to accommodate large 
differences between components of rf d ^ b or between combinations 
thereof might be infeasible, e.g. due to velocity constraints related 
to the agents or their individual tasks. The process (20), (21) is 
therefore assumed to be subject to locally bounded regions of 
attraction. Such boundedness is similar to the range of frequency 
detuning Aa> in (14), which limits stable phase synchronization. 

Note. Normalization is preserved e.g. by adjusting the compo- 
nents of d d ^ b such that d/ =0 holds, which is the 
derivative of the normalization constraint. 

In the following, we outline a possible realization of the process 
(20), (21) featuring the mode illustrated in Fig. 6(b). In this mode, 
the event pairs (£§,?()) and (i\,t b ) appear synchronized simulta- 
neously. The former is readily synchronized by the phase 
dynamics (12), (13) employing the stable equilibrium point 
'I'r.e = 0. In order to additionally synchronize the latter, we design 
the entrainment of d d according to 

d l = —d 2 , (22) 



Ad sin V 

"2,h "2,1 



The thresholds and d'^ define the lower and upper bound 
on the entrainment of d\. Assuming isotropic coupling between 
the agents, the entrainment of d b is designed analogously. 

Transformation between Movement and Phase 

The synchronization process developed in the previous section 
governs the phase variables (F^it) as well as the relative primitive 
durations d' l ^ b (t). Since we target the integration of the 
synchronization behavior in the perception-action loop of robotic 
agents, the movement trajectories need to be transformed on-line 
into the process variables and vice versa. 

From Movement to Phase 

The problem considered first is how to determine the partner's 
phase instantaneously, based on measurements of the movement 
trajectory. Besides the instantaneous phase 6(t), the solution 
presented in the following also provides event predictions ?/„■, and 
thus via (2) and (3), predictions of the relative primitive durations 

d(t). 

Existing methods and open issues. Different methods have 
been applied to extract instantaneous phase variables from limit 
cycles that are known only by their observables, e.g. their cyclic 
movement trajectories. However, none of them fulfills our 
requirements entirely. First and foremost, only one-dimensional 
and narrow-band trajectories can be analyzed properly by the 
common methods. These methods are the analytic signal concept 
based on the Hilbert transform [42] and the state space methods 
[51] retrieving the trigonometrical phase angle in a position- 
velocity state space. Moreover, the former is restricted to off-line 
analysis, see [6] and [52] for a comparative discussion. The 
technique of linear phase interpolation between single marker 
events per period [42] can be considered analogous to the analysis 
of return times on the Poincare map. Since this technique is 



(t a t b ) 




Figure 6. Circular illustration of the synchronization problem between two limit cycles. The exemplary limit cycles /' (inner circle) and y b 
(outer circle) are introduced in Fig. 4. (A) The degrees of freedom available for synchronization: The periods T"l h and the phase difference O are both 
governed by the process (12), (13). The relative primitive durations rf a//b are governed by the process (20), (21). (B) Perfect synchronization of the 
event pairs (t^,^) and (t\,t\), leading to coincident circles and events. 
doi:1 0.1 371 /journal.pone.00951 95.g006 
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applicable regardless of the frequency components and the 
dimensionality of the analyzed trajectory, we adopted our phase 
definition accordingly. However, the following challenges remain, 
preventing to calculate the phase straightforward via (4): 

• Movement variabilities due to interaction or other perturba- 
tions considered as noise will cause the limit cycles of human 
partners to be non-stationary, i.e. the period T(t) and thus, also 
the relative primitive durations d(t) become instantaneous 
variables. 

• The variables T(i) and d(t) refer to a parameterization of the 
current period as a whole. Hence, on-line applications require 
estimates that are continuously predicting the future values 
these parameters take at period completion. 

The instantaneous phase of non-stationary limit 
cycles. The desired phase variable is required to instantaneously 
reflect changes of the period, while it is also required to comply 
with the definition (4) prescribing the unperturbed phase 
evolution. Given a prediction of the event tL,i{t) denoting the time 
of completion of the current period i, we propose a phase estimate 
for the time fQ„< * <?z.,j(0 given by the solution of 



Prediction of events from observation. Both the instanta- 
neous phase (25) denoting the numerical solution of (24) and the 
relative primitive durations obtained from (2) and (3) require on- 
line predictions of the events ?/„-, /= l,...,L in the current period i. 
To that extent, we assume the state £ to be fully observable up to 
time t. The task-related segmentation points S=[£j ... ££\ are 
assumed to be known and constant. 

We propose the following two-step technique to obtain 
predictions from experimental measurements: 



Acquiring limit cycles: Reference limit cycles 



/ : with fe[f 0 f L ] 



(27) 



are acquired over single, complete periods. A family of limit 
cycles y' , r= 1, ...,R is built from a number of R cycles. These 
feature differing periods T covering the expected range of 
periods, see example in Fig. 8(a). 

Classifying limit cycles and predicting events: The current state £ is 
classified with respect to the family of reference limit cycles. 
First, the similarity to each y r is determined by the respective 
minimum of the distance metric 



cb(t)-- 



2ni-0 
h,i(t)-t 



(24) 



which is a linear differential equation with time-varying coeffi- 
cients. The initial condition reads 6(to,i) = 2n(i— 1). Time-varying 
predictions of the event tL,i(t) are instantaneously reflected by the 
phase velocity (24), see example plot in Fig. 7. Numerical 
integration of (24) yields the phase trajectory 



(28) 



with Q being a n x n positive definite weighing matrix. Next, 
the closest cycle y r is selected by 



r* = argmin Acf. 



(29) 



0(0 = 



Co(T)dT+0(%), 



(25) 



which is due to &>(f) > 0 monotonically growing. For times 
;->?£„(;), the solution of (24) converges to B = 2ni. 

Note. Given a stationary limit cycle and assuming perfect 
prediction ?/,„■ = ?/.„■ = const., the solution of (24) can be derived 
analytically. It reads 



0(0 = 



2tt 



tL,i — to,i 



(t-t 0 ,i)+e(to,d, 



(26) 



which is obviously the harmonic angular phase complying with 
definition (4), cf. gray graphs in Fig. 7. 




Figure 7. Instantaneous phase calculation. (A) Exemplary evolu- 
tion of the predicted event /£„ overtime t. (B) Corresponding evolution 
of the phase 8 obtained from (25). The slope of 8 instantaneously 
relates the left over phase 27n — 8>0 in period / to the left over time 
span ?£„■ — 1>0. Black dots denote boundary conditions. Gray graphs 
depict perfect prediction and the harmonic phase respectively. 
doi:1 0.1 371 /journal.pone.00951 95.g007 



If the state £ is close to the segmentation points, the distances 
A£ r are nearly equal. In this case, undesired switchings of r* are 
avoided by switching from previous r*' to current r* only if a 
certain threshold 



A^<Ar'(0-Af(0 

is exceeded. Finally, predictions of any future event ?/„■ at time / 
are obtained from 



--t+f, —f , with fj >f 



(30) 




Figure 8. Classification-based event prediction. (A) Family of 
R = 4 limit cycles / with differing periods T r . In the position-velocity 
state space, shapes differ due to x scaling with V . (B) Close-up 
illustrating distance-based classification (top). Events are predicted 
based on the acquired evolution of events in /* (bottom). 
doi:10.1371/journal.pone.0095195.g008 
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where f, denotes the corresponding event in "f and f the time 

at minimum distance Af in /' , see Fig. 8(b). 

Note. The quality of the event predictions depends on the 
number of reference limit cycles and their distribution of periods, 
i.e. how fine-grained the covered portion of the state space is 
sampled. 

From Phase to Movement 

Robotic agents implementing the synchronization behavior 
require the transformation inverse to the previous one as well. By 
means of a technique based on movement models, the process 
variables are transformed back to the cyclic movement trajectory 
representing the individual action task. After defining the required 
model properties, we develop an exemplary realization of this 
transformation through a model based on the minimum-jerk 
criterion [53]. It renders human hand movements in goal-directed 
tasks [54]. 

General movement model. The trajectory is again com- 
posed by a given number of L primitives l=\,...,L connecting 
the segmentation points with relative primitive durations d/. 
Inverse to the phase-amplitude decomposition of the cyclic state 
trajectory, we require the movement model to take the general 
form 

z=md,s). (3i) 



define the segmentation points and £/, since (33) implies 

i/_ i =xi = 0. For any choice yj<x.t, (32) minimizes the jerk x . 
Re-parameterization of the minimum-jerk model. The 

parameter of the /th primitive (32) is substituted by the process 
variables 8 and d, i.e. 

Xi=hi{B,d). (34) 

If (34) fulfills the condition 

hi(8,d) = l, (35) 

the subsequent primitive is activated, i.e. the transition /i/h->/( /+ i 
and n L >-*fli respectively is triggered, see Fig. 9(a). The substitution 
hi in the current period / is realized by 

WA=2^si[0-Wi-ud], (36) 

which is composed as follows. The phase value 0(//_i„) obtained 
from (11) is subtracted to remove the offset at the event of 

primitive entry f/_i„-. The factor - — — scales phase values 

2ndi 

ee[0(ti^ Ui ),6(tl,d] to values X/e[0,l]. The term 



The function / denotes a mapping of the phase 6, the 
distribution d = [d\ ... d^ , and the task-related segmentation 
points 3=[£j ■■■ £,l[ onto the continuous state trajectory £. In 
brief, an appropriate movement model needs to. 

• fulfill the condition (1) for finite periods T, 

• facilitate temporal scaling implemented by 6 and d, 

• facilitate spatial scaling depending on S. 

Models complying with these properties are discussed in [27]. In 
the following, we re-parameterize a model £,=f{t) explicidy 
depending on time t to comply with (31). 

Note. The process variables 0 and d implement the degrees of 
freedom available for the voluntary behavior of movement 
synchronization. The movement model f has to necessarily comply 
with the task-related segmentation points E. 

The minimum-jerk model as an example. Human hand 
trajectories composed of point-to-point movements are known to 
be successfully reproduced by the minimum-jerk model formulat- 
ed in a Cartesian frame [53]. With reference to the human-robot 
experiment described later on, we investigate this polynomial-type 
model. The state i=[x x] T is defined, with x and x denoting the 
hand (effector) position and velocity in a Cartesian frame. The 
movement model (31) is then realized by a sequence of L point-to- 
point primitives 



l l l ■ X=f,(xi) = (xi-Xi-i)g(Xi)+Xi-u 



(32) 



parameterized by jfte[0,l]- The function ge[0,l] denotes the fifth- 
order polynomial 



g(7j) = (>7]-l5xt + W 7 ]. (33) 
The start point JC/-1 and the end point xi of the primitive ji/ 



si = ^-j— j — , with d 0 

1 - £/=o d 'i 



= 0. 



(37) 



ensures, that the boundary condition hi(2ni,d) = 1 is fulfilled for 
any time-varying d. Witii d'j we denote the actual value that dj 
assumed at past transition Hj^Hj + l ■ 

Note. If d = const, holds, s/ = 1 is satisfied, and the substitution 
(36) becomes piece-wise linear, i.e. ^/ccfj. If additionally 9 = const. 




Figure 9. Transformation of the process variables 0,d into a 
limit cycle with L = 3 primitives ft. The minimum-jerk movement 
model is employed. (A) Piecewise-continuous substitutions hi illustrated 
for the unperturbed phase with 8= const, (gray graph) and d = const. 
(B) Continuous, cyclic movement trajectory composed by polynomials 
//. For the corresponding limit cycle representation, cf. y h in Fig. 4A. 
doi:10.1371/joumal.pone.0095195.g009 
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holds, piece-wise linear / oc t is obtained. Thus, if the synchroni- 
zation process is in steady state, the trajectory £(7) is composed by 
minimum-jerk movement primitives, cf. example in Fig. 9. 

Human-Robot Synchronization Experiment 

The concept of movement synchronization is applied to render 
the interactive behavior of a robotic agent that performs a joint 
action task together with a human partner. Supporting informa- 
tion is provided in Video SI. The human-robot synchronization 
experiment fulfills two goals. First, it provides a proof-of-concept 
implementation successfully illustrating the developed synchroni- 
zation behavior by means of a robotic interaction partner. Second, 
it serves to explore the potentials of the developed robotic behavior 
in joint action tasks with human interaction partners. 

In the following, superscripts 11 and b are replaced by H and R 
when variables belonging to the human and the robotic agent need 
to be distinguished. 

The Joint Action Task 

The design of the experimental task is inspired by the dot- 
tapping paradigm studied in our previous work [5], [6]. The 
following points summarize the desired features: 

• Both agents perform repetitive movements composed by 
sequences of multiple primitives with closed trajectories (cycles). 
Multiple cycles performed consecutively allow to study 
synchronization effects. 

• Since we investigate different modes of synchronization, the 
cycles need to offer potentially relevant synchronization events. 

• The task is goal-directed, i.e. the agents' effectors have to reach 
one or more goal points. 

• Overlapping workspaces provoke close interaction and constrain 
synchronization, since collision avoidance is required in certain 
workspace regions. 

• Mutual pick up of sensory information about each others' actions 
is allowed to let interaction emerge. 

Accordingly, the task paradigm depicted in Fig. 10 is designed. 
Both the human and the robot perform cyclic sequences of 
multiple movement primitives with their right arm/manipulator, 
while sitting opposite to each other at a round table. The task is to 
carry barbell-shaped objects from pick points to place points which 
are marked on the table. The objects have a height of 140 mm and 
a weight of 0.19 kg; they are equipped with an iron sheet and a 
plastic disc on top with reflective markers attached, allowing for 
magnetic grasping and marker-based tracking respectively. The 
participant wears a glove with an additional weight and markers 
attached. Total weight of the glove is 0.51 kg. Its purpose is to 
naturally slow down the humans' movements. The agents' 
workspaces are arranged such, that two objects can be exchanged 
between them in a cyclic fashion. Within each pick-and-place 
movement, the table shall be touched at a tap point close to the 
agent. The robot only performs a tap when carrying an object, 
hence the agents' movement cycles differ. A human-size mobile 
robot with anthropomorphic arms serves as the interaction partner 
in the experiment, see Fig. 1 1 . 

Three synchronization modes are investigated in the above joint 
action task, see Fig. 12. These modes synchronize different 
combinations of pick, place and tap actions. Since the objects can 
be exchanged by sequential pick-and-place actions, the modes 
comply with the task-related goals. Note, that each of the 
segmentation points features two events, entry and leave of the 
respective point. These frame the so-called dwell time, which is 



Robot 



1.15m 




O tap R O Pick R, Place H O Place R, pick H Q ta P H 



Figure 10. The joint action task designed for the human-robot 
synchronization experiment. In a symmetric setup, both human 
and robot perform slightly different action tasks while facing each 
other. Odd-indexed primitives /(, consider dwell times, even-indexed 
ones denote movements. Target points are marked by circles of 
115 mm in diameter. 
doi:10.1371/journal.pone.0095195.g010 

known to be part of human motor control in aiming tasks [55]. 
The above modes are represented by stable equilibrium relations 
that are featured by the unified synchronization process, see 
Table 1 . Details on the data acquisition system, the robotic system 
[56,57] and the implementation of the synchronization behavior of 
the robot are given in the Appendix S 1 . 

Participants, conditions and procedure 

Participants. Procedures were approved by the ethics 
committee of the medical faculty of the TUM and conformed to 
the principles expressed in the Declaration of Helsinki. In total, 1 2 
people (9 female) participated in this experiment. They were 
between 20 and 48 years old (M = 30.8). All were right handed and 
had normal or corrected-to-normal vision and were nave as to the 
purpose of the experiment. For participation, they were paid 
8 EUR per hour. Prior to their inclusion in the present study, all 
participants gave written informed consent. The individual in this 
manuscript has given written informed consent (as oudined in 
PLOS consent form) to publish these case details. 

Conditions. Two conditions manipulated the synchroniza- 
tion behavior of the robot: 

• NOS: No Synchronization. 

The robot performed at f) = <u R , with constant frequency 
co R = 1.3rads~ . Its relative primitive durations were set 
constant to </ R = [0.05 0.2 0.05 0.45 0.05 0.2] T . 

• PES: Phase and Event Synchronization. 

The robot aimed to synchronize the three modes we designed 
above, applying the parameters from NOS and the coupling gains 
vr = 0.3rads _1 and K d = 0.028-' . 

In both conditions, the effector trajectory of the robot was 
subject to collision avoidance as described in the Appendix SI. 

Procedure. The experimental procedure was as follows. The 
mobile platform of the robot was maneuvered to a target pose 
calibrated with respect to the table by means of markers, such that 
the goal points assigned to the robot were within the workspace of 
its right manipulator. Similarly, the participants were seated in a 
comfortable posture close to the table, cf. Fig. 10. A written 
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Figure 1 1. Experimental setup. Left: The scenario of a prototypical joint pick-and-place task. Right: Hand movements are made available to the 

robot in real time by tracking the glove the human interaction partner is wearing. 

doi:10.1371/journal.pone.0095195.g011 



instruction handed to the participants provided the description of 
the human-robot joint action task. In particular, the participants 
were advised that for the task to be successfully fulfilled, joint 
action in cooperation with the robotic partner is required. In order 
to provoke natural interaction, they were instructed to perform at 
comfortable speed and to touch the marked positions precisely in a 
single movement. Direct hand-over and sliding the objects over 
the table was not allowed. The participants were neither informed 
about the synchronization behavior of the robot, nor were they 
advised to synchronize. At the beginning of each trial, they were 
asked to rest with an object in their hand in the respective tap 
position and instructed to start executing the task as soon as they 
heard an acoustical start signal (high-pitched tone) through their 
head phones. The stop signal (low-pitched tone) was presented 
after they had performed ten cycles. The start signal was timed 
such that the modes described in Fig. 12 were provoked initially, 
i.e. for mode 1, both the participants and the robot were triggered 



tap 



place 



tap 



pick 



T-? ¥-4 — ¥-4 — t-4 J- 

d H d H d H d H d H d H d H d H 
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Figure 12. Evolution of events in the experimental task 
synchronized in three modes. The relative durations d/ correspond 
to the primitives /(, defined in Fig. 10. Again, odd-indexed durations are 
due to expected dwell times in the segmentation points. (A) The cycle 
y H of the human. (B) The cycle y R synchronized to y H in three different 
modes, denoted mode 1-3. Vertical dashed lines indicate synchronized 
events. Intuitively speaking, the human precedes the robot in mode 2 
and vice versa in mode 3. 
doi:10.1371/journal.pone.0095195.g012 



simultaneously being in their tap points, for mode 2, the robot was 
triggered when the participants entered their place points, and for 
mode 3, the participants were triggered when the robot entered its 
place point. Six sets (two synchronization conditions x three start- 
off modes) each consisting of three trials were performed which led 
to a total of 1 8 trials. These sets were carried out in a randomized 
sequence of two blocks, each with three sets under the same 
synchronization condition. The sets manipulating the start-off 
mode were presented in randomized order in each block. 

Quantitative Measures 

The following measures are deployed to assess the synchroni- 
zation behavior observed in the experiment. 

Event synchronization. The synchronization of events 
targeted by the behavioral model of the robot is assessed based 
on the measured Cartesian position trajectories of the human 
hand x H (t) and the robot effector x R (/). Those are recorded 
simultaneously by the motion capture system, thus differing 
processing delays are eliminated. Trajectory segmentation and 
event extraction is identical with the implementation of the robot. 
According to the definition of event synchronization above, we 
calculate for each synchronization mode m the temporal lags 
within all event pairs (?JJ,?/r) with the indexes / H / R chosen 
corresponding to the events synchronized in mode m. For each 
mode m, the lag magnitudes are averaged per period ! H , i.e. over 
event pairs with rjjjelfj'j,^). Those averages provide contin- 
uous measures of asynchrony, which we denote ASYN OT ; H. In 
each period i H , the best fitting one out of the three modes is 
detected by selecting the smallest asynchrony. The per-trial 
average of the latter over all periods 7 H reads 



MASYN= F ^min ASYN m . H , 



(38) 



which we call the mode-related asynchrony. 

Note: The mode-related asynchrony quantifies the mean time lag 
between multiple event pairs measured in seconds. Only complete 
sets of event pairs corresponding to the defined modes are probed. 

Mode distribution and mode switches. At any time, one 
of three synchronization modes is considered to be active, and 
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Table 1. Stable equilibrium relations of the synchronization process. 





Mode 


Phase difference <D r , e ,m 


Relation of relative durations d H,R 


m-\ 


$r,e,l =0 




df = df, df = df, df = df, df = df, df = df 


m = 2 


4 , r,e,2=27t^ 




df =df, df=df, df=df, df = df, df=df 


in = 3 


Or,e,3= 2 ' r X 




df =df, df=df, df=df, df = df, df = df 



doi:10.1371/journal.pone.0095195.t001 



pursued by the robot in condition PES. According to the vector 
field design cf. Appendix S 1 and Table 1 , we determine the active 
mode 



relative primitive durations d ' is essential to the synchronization 
of multiple event pairs. It is assessed by the root-mean-square error 
defined as the residual 



1, if O r < 



©r,e,l+fr,e,2 



V0> r > 



O r , e , 3 + 2ti 



m = , . ^i+O^ ^2+^3 (39) 

2 2 

3, otherwise. 



Given the evolution of the active mode m(t), we analyze the 

relative distribution of modes — — as an indicator of the within- 

N 

dyad preferred synchronization mode, where N m is the number of 
samples in active mode m and N the total number of samples per 
trial. Note that the number of samples is representative of the 
continuous amount of time spent in a certain mode. Furthermore, 
the temporal persistence of modes is measured by the number of 
mode switches, i.e. the number of samples {j \ m(t f) # m(t y + 1)} 
per trial. 

Synchronization index. Phase synchronization is often 
quantified by means of the synchronization index, see e.g. [43] 
for a comprehensive review. Given the time series of the phase 
difference fl>(?) consisting of N directional observations ®(f/), the 
synchronization index 



SI = 



N 



,Wtj) 



t-cv 



(40) 



is calculated, where CV denotes the circular variance of an 
angular distribution. The synchronization index SI is also called 
mean phase coherence. The synchronization concept in this article 
introduces multiple modes, represented by differing equilibrium 
phase differences. Trials with one or more mode switches would 
heavily degrade the index (40). Hence, we propose to calculate the 
synchronization index separately for epochs of the same active mode. 
The resulting indexes SI„, are then combined per trial into the 
mode-related synchronization index 



MSI = 



N 



J2 N >» S1 " 



(41) 



weighted by the respective number of samples N m . 

Note: The MSI lies in the interval [0,1]. Given a perfectiy 
uniform distribution of it would equal zero. It equals one 

only if the synchronization process is persistendy in steady-state, 
which means that all samples of 0(?) point to the same direction. 

Entrainment error of relative primitive durations. As 
shown in our synchronization concept, the entrainment across the 



RMSE($ =d%)- 



N 



7 = 1 



(42) 



with the primitive indexes / H / R chosen corresponding to the 
equilibrium relations summarized in Table 1. For each relation 
and epoch of the same active mode m, the entrainment errors are 
obtained from (42) and averaged over the five mode-dependent 
equilibrium relations afterwards, yielding the errors RMSErf, m . 
Analogously to the above definition of the mode-related synchro- 
nization index (41), those are then combined by the weighted 
average 



RMSE rf = - Y yV,„RMSE rf „ 



which assesses the overall entrainment error of rf H ^ R . 



(43) 



Experimental Results 

The observable degree of event synchronization between the 
movements is evaluated as external measure. Feedback gathered 
from a short questionnaire is reported as well. We also assess the 
synchronization behavior through measures relying on internal 
variables of the robot. Note, that the results presented in the 
following are based on a group of nine participants unless stated 
otherwise. The remaining group of three participants performed at 
movement speeds either far below or above the speed range the 
robot is capable of moving at, thus impeding movement 
synchronization in the experiment. Possible reasons are discussed 
later. 

External Assessment of the Synchronization Behavior 

The following results allow to explain, how far the overall goal 
of our synchronization concept is reached objectively, i.e. if it 
fosters the entrainment of movements by synchronizing multiple 
event pairs. In addition, subjective feedback from the participants 
gives rise to discuss some perceived effects. 

Subjective reasoning. After having completed the experi- 
ment, participants were asked whether or not they had the feeling 
that the robot reacted to them. In case of a positive answer, they 
were asked to state if they found that perceived reactiveness 
pleasant (yes/no) and to reason about this answer. Eleven out of 
twelve participants recognized reactiveness of the robot in 
response to their movements during parts of the experiment. 
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Ten out of eleven participants who answered positively stated that 
they liked the perceived reactiveness, giving reasons such as: 

• It makes the robot appear lively. 

• Having the control over task speed is pleasant. 

• Adjustment towards similar speed is pleasant. 

• It fosters smoother interaction. 

• Negotiation among partners is beneficial. 

• It is a nice feeling, but a bit uncanny as well. 

The one who disliked the reactive behavior of the robot 
described the interaction as flurry and unsteady. 

Event synchronization. The evaluation of the objective 
measure of event synchronization, which is the mode-related 
asynchrony MASYN, is depicted in Fig. 13. A 2x3 repeated 
measures ANOVA with the within subject factors condition (NOS, 
PES) and start-off mode (1-3) reveals a clear decrease of asynchrony 
in each of the start-off modes, .F(l,8)= 18.06,p = .003, if the robot 
applies synchronization behavior, i.e. the condition PES. Irre- 
spective of the synchronization condition, start-off mode 1 
numerically results in lowest asynchrony values, whereas a slight 
trend towards increased asynchrony is visible for mode 2 and 3. 
However, differences between start-off modes were not significant 
and no significant interaction effect was observed, both p > .4. 

Internal Assessment of the Synchronization Behavior 

In the following, the behavioral dynamics is evaluated based on 
its internal representation, i.e. the internal variables of the robotic 
agent. 

Entrainment of phases and relative primitive 
durations. To start, we explain the inner processes governing 
the synchronization behavior of the robot during an exemplary 
trial. The trajectories of relevant process variables are illustrated in 
Fig. 14. After starting off in mode 3, cf. initial phase difference in 
Fig. 14B, the relation df-=d^ is entrained amongst others, see 
very left part in Fig. 14A. Note that the attractor landscape 
generated by the vector field H is morphed depending on the 
entrained components of d R . Thereafter, the phase velocity of the 

robot f) is slowed down by the function c(Ax) due to collision 
avoidance, Fig. 14C. As the participant progresses fast, the robot is 

forced into mode 1 . Through modulation of 0 within the tuning 
range [l,1.6]rads , which is defined by its natural frequency cu R 
and coupling gain K, the robot attempts to sustain the mode it is 
close to. It can be seen, that now the relation df = is pursued. 
After a while, the participant again increases speed, which leads 
the robot to finally switch to mode 2. Here, the relation d R = d^ 
becomes entrained. 



0.4 

z 

g0.2 

nl 1 L 



I NOS I I PES 
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Figure 13. The mode-related asynchrony MASYN. Values are 
averages over all trials for the three start-off modes under the 
conditions NOS and PES. The bars represent standard errors of the 
mean. 
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Figure 14. Evolution of selected process variables for a sample 
trial under condition PES and start-off mode 3. Vertical solid lines 
denote mode switches. (A) The duration df(t) of the robot entrained 
with one of the durations d^ 4S (t) of the human, depending on the 
active mode. (B) The relative phase difference O r (/), and the vector field 
H with its time-varying attractive regions (dark) and repulsive regions 
(bright) representing the modes m= 1,2,3. (C) The robot phase velocity 

f? R (0 and collision avoidance function c(Ax). 
doi:10.1371/journal.pone.0095195.g014 



Preferably synchronized modes. The relative amount of 
time spent in the synchronization modes and the relative amount 
of mode switches are illustrated in Fig. 1 5 . Here, the relative time 
spent in each mode provides an intuition of how long, on average 
and with respect to the trial durations, each mode has been active 
within the robot behavior. It can be seen that under PES, the 
relative share of that mode increases, which the human-robot dyad 
has started with (upon trigger). To access the differences between 
NOS and PES with regard to the amount of time spent in 
triggered mode, planned comparisons were performed between 
conditions (NOS, PES) within the respective start-off mode. If 
participants were triggered to start off in mode 1, the relative 
amount spent in mode 1 is significantly higher under PES 
compared to NOS, t(S)= - 1.90, p = .047. Since under NOS, the 
robot only observes but not actively pursues these modes, that 
increase is due to robotic synchronization behavior in PES. Similar 
results were obtained for start-off mode 3, ?(8)= — 2.57, p = . 017. 
However, the difference between relative mode share in PES and 
NOS during start-off mode 2 was only found to be numerical, 
z=— .77, p> .2. Mode 2 was also the dominant mode during 
NOS. Hence, no effect of the synchronization behavior is visible 
here. Overall this shows that when being triggered close to the 
attracted modes, the robot successfully sustains them. 

This is also reflected by the relative share of mode switches. 
Results of a 2 x 3 repeated measures ANOVA on condition and 
start-off mode show that the amount of mode switches decreased 
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Figure 15. Relative amount of time spent in each mode and 
relative amount of mode switches. Both are averaged separately 
over all trials for the three start-off modes under the conditions NOS 
and PES. 
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under PES in each of the start-off modes i 7 (l,8)= 17.83, p = .003. 
Neither a difference between start-off modes nor an interaction 
effect was observed, p>3. 

The preferred phase relations as a result of phase synchroni- 
zation are reflected by histograms of the phase difference, see 
Fig. 1 6 left, which is a representation complementary to the mode 
distributions above. Some preference towards certain phase 
relations can be recognized even under condition NOS, which is 
ascribed to human synchronization attempts due to the static 
behavior of the robot. Under PES, the distribution gets sharpened, 
forming three distinct peaks. When comparing that distribution in 
Fig. 1 6 left with the distribution of actively attracted equilibrium 
points in Fig. 1 6 right, their coincidence indicates successful phase 
entrainment through the robot behavior. Weight on the peak 
corresponding to mode 2 (i.e. O r ,e,2) is strongest, followed by the 
peak at mode 1 (i.e. O r ,e,l), which is in line with the distribution of 
modes in Fig. 15. Note that the smeared distributions of <J5 r ,e,2 an d 
Q>r,e,3 are due to their dependency on the relative primitive 
durations d R (t). 

Quantitative assessment of the synchronization 
process. The convergence and performance of the dynamical 
process of synchronization is measured by means of the process 
variables, which are the phases or the phase difference O 
respectively, and the relative primitive durations d H ^ R . The results 
are illustrated in Fig. 17. To access the differences between NOS 
and PES governed behavior, 2x3 repeated measures ANOVAs 
were performed with the within subject factors condition and start- 
off mode. For MSI, the condition PES causes an increased 
entrainment compared to NOS, _F(1,8) = 25.73, /> = .001, see 
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Figure 16. Relative frequencies of occurrence of phase 
differences. (A) The relative phase difference <5 r under the conditions 
NOS and PES. (B) The attracted equilibrium phase differences O rc under 
PES. 
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Figure 1 7. Entrainment measures. Values are averages over all trials 
for the three start-off modes under the conditions NOS and PES. The 
bars represent standard errors of the means. (A) The mode-related 
synchronization index MSI. (B) The root-mean-square error of durations 
entrainment RMSE rf . 
doi:10.1371/journal.pone.0095195.g017 

Fig. 1 7A. Between start-off modes no significant difference was 
observed, p > .5. Also no significant interaction effect was detected. 
Similar results are obtained for the entrainment errors of 
durations, which are decreased by the entrainment process under 
PES, F(l,8) = 36.57, p<.00l, see Fig. 17B. Lowest errors with 
respect to the attracted equilibrium relations are achieved in start- 
off mode 1 under PES, as shown by a significant interaction effect, 
F(2,l6) = 5.29,p = .0n. 

Instantaneous phase estimation. The characteristic evolu- 
tion of the period and phase estimation obtained from the human 
movements are illustrated by means of the sample trajectories 
depicted in Fig. 18. The events result from on-line 

segmentation of the movement trajectory J H (0, see Fig. 18A. 
Those events denote the time of the human hand entering the tap 
point, and the completion times of the periods i. The instanta- 
neous period T H (i) depicted in Fig. 18B is equivalent to the 
prediction i$,i(t), due to the definition of the instantaneous 
period T a (f)= t^j (t) — . For comparison, the values Tf 1 
measured at period completion are shown as well. Note that due to 
the finite number of reference cycles used for event prediction, 
7" H (?) is not continuous. More specifically, when the reference 
cycle selected by classification switches, corresponding event 
predictions switch as well. It can be seen that the on-line 
estimation of the human phase f) H (/) successfully satisfies our 
demands: It reflects changes of T H (/) instantaneously and 
smoothly, while it still remains 27t-periodic with respect to the 
events f^,,- marking the period completions. 

Discussion 

The results gained from the human-robot synchronization 
experiment provide the proof of concept and evidence the 
potentials of synchronization behavior in human-robot joint 
action. In brief, the following novel insights beyond existing 
research on movement synchronization are identified: New 
synchronization modes are explored in the context of goal- 
directed joint action tasks. The mode-related asynchrony MASYN 
is successfully decreased by the proposed unified entrainment 
process of both phase and relative primitive durations. Therefore, 
the interactive behavior of the robot driven by the proposed 
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Figure 18. Evolution of the instantaneous phase estimation for 
the first half of the sample trial. (A) The ^'-component of the human 
hand position, and the events t% ti . (B) The estimated instantaneous 
period f H (t), the measured period Tf 1 , and the estimated phase 6 H (t) 
taken modulo 2k. 
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concept under the condition PES significantly improves the overall 
degree of synchronization between the robot and human partners 
compared to the static behavior under NOS. Mixed discrete and 
continuous repetitive movement primitives are synchronized in the 
pick-and-place task. Hence, the novel phase estimation technique 
is evidently applicable to multi-primitive movement cycles, which 
cover a wide range of repetitive joint action tasks. Subjective 
feedback reveals that the synchronization attempts of the robot 
towards the designed modes lead to an enriched sense of 
interaction with the robot for most of the participants. This 
highlights the strong potential of this approach to advance the 
social interaction capabilities of robots that perform joint actions 
with humans. In the following, the synchronization concept and 
the experimental results are discussed in more detail, both in the 
light of human-robot joint action and the design of interactive 
behavior for artificial agents. 

Implications on Human-robot Joint Action 

Both the objective improvement of event synchronization 
achieved in our exploratory study and the summary of subjective 
feedback underpin the endeavor to investigate synchronization 
behavior evident in human-human interaction in the context of 
human-robot joint action. The behavioral dynamics pursues weak 
phase synchronization enforced by sinusoidal coupling of strength 
K = 0.3 rads~ , which is close to the coupling strengths of 
uninstructed human-human movement synchronization identified 
in [6]. Thus, the applied weak forcing is such that participants 
could not only switch between synchronization modes, they also 
could have easily resisted or distorted synchronization within the 
constraints imposed by the hand-overs. In support of this, for a 
group of three participants we observed that, under PES, the 
degree of synchronization deteriorated which stands in contrast to 
the reported improvement of synchronization for the group of nine 
participants. Most participants appreciated the synchronization 
attempts of the robot. In short, their answers let us conclude that 
they had an enriched sense of interaction. 

However, the objective results also suggest some implications 
and pitfalls that need to be addressed carefully in the design of 
synchronization behavior. Subjectively pleasant, mutual entrain- 
ment of movements appears to be rather sensitive to parameter- 



izations of the behavioral dynamics, first and foremost their 
attractors and their associated strengths. If those do not match the 
individual entrainment behavior of the human counterpart within 
certain ranges, inter-agent entrainment may fail and even worse: 
artificial entrainment attempts may be misinterpreted and lead to 
a degraded sensation of interaction compared to non-reactive 
behavior. 

The appearance of the robotic partner and, strongly connected, 
its capabilities anticipated by the participants, is expected to also 
affect human interactive behavior [58]. Besides its manipulator 
kinematics having similarity to the human arm, the design of the 
robotic agent used in our study is rather associated with functional 
and technical attributes, than with humanoid ones, see Fig. 1 1 . 
Moreover, we did not brief the participants on the behavior they 
could expect from their robotic partner. One of the participants 
reported to perceive some uncanny-ness when facing the reactive 
behavior of the robot, which could likely be originated by the 
potential mismatch of rather crude appearance and sophisticated 
interaction capabilities. Both factors give rise to further investiga- 
tions going beyond the scope of this study. 

Human interactive behavior may furthermore heavily depend 
on how the task context is conveyed and understood [59]. Human- 
robot experiments are usually conducted within controlled 
laboratory settings, which makes it hard to reliably create the 
desired context in the participants' minds, e.g. that of an everyday 
activity performed within familiar surroundings. While the 
implemented behavior is geared towards the abstract context of 
joint action, emphasis on the individual performance requirements 
and the cooperative aspect of the task is likely to vary between 
participants. For example, the instruction to precisely hit 
designated tap points might be assigned higher priority than an 
uninstructed and likely unconscious desire to reduce dwell times of 
the partner in favor of smooth and fluent interaction. All of the 
above discussed points may affect uninstructed, i.e. emerging 
synchronization behavior in human-robot joint action. We 
hypothesize that among those points reasons can be found for 
the hampered synchronization and behavioral mismatch we 
observed within the disregarded group of three participants. 

Design Issues Regarding the Synchronization Concept 

One key idea of the synchronization concept is the design of 
synchronization modes by means of a dynamical synchronization 
process unifying both phase synchronization and the entrainment 
of relative primitive durations. It has to be emphasized that both 
processes are usually cross-coupled: The entrained components of 
rf''/ b depend on the attracted mode m, cf. Table 1 defining the 
modes implemented in the experiment. Changes of the relative 
• a/b 



due to mutual entrainment cause shifts of 

a/b 



primitive durations d 

the equilibrium points <J5 C ' within the phase dynamics on the one 
hand. On the other hand, the attracted mode m is determined by 
the equilibrium point which is closest to the phase difference <I>. 
Depending on the designed modes and their attractor dynamics, 
the interaction of both processes might not be generally stable by 
itself, and therefore eventually result in oscillations between 
attracted modes. By defining reasonable bounds d*J^ and choosing 
the gain Kj « K, the entrainment process of durations is bound to 
certain attractor regions and slow compared to the phase 
difference dynamics. Though we did not encounter that kind of 
instability in our experimental setting, the formal derivation of 
stability bounds remains an open issue. 

The presented design of synchronization behavior offers several 
interesting degrees of freedom which are not investigated in this 
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article. The structure of the phase synchronization process is 
originated from the extended Kuramoto model [6] and variants of 
the HKB model [14] respectively, which evidently replicate 
human synchronization behavior. In contrast, the implemented 
entrainment structure of the relative primitive durations is 
considered prototypical, leaving room for further investigation 
and validation in the field of human-robot joint action. Similar to 
the phase dynamics of the HKB model, synchronization modes 
can be assigned differing weights through variable strengths of 
attraction. Another degree of freedom is provided by the natural 
frequency. In line with oscillator theory, the natural frequencies of 
the harmonic oscillators govern the individual behavior, since they 
autonomously drive the agents' task progress at their individually 
desired speed. The domains of successfully negotiated entrainment 
between the agents is defined by the frequency difference. 

Since the definition of the instantaneous phase purely depends 
on recurrent events within the period, the movement trajectory 
can be of arbitrary shape, as long as predictions of those events are 
provided. Instead of the presented technique based on minimum 
distance classification in the state space, the application of machine 
learning techniques such as programming by demonstration [60] 
could be investigated alternatively, for the sake of a flexible 
encoding of observed movement sequences and event predictions. 

Summary and Conclusion 

In this article, we propose a novel concept and design 
methodology to synthesize goal-directed synchronization behavior 
for robotic agents in repetitive joint action tasks. Those tasks are 
assumed to be performed by dyads of agents in a common 
interactive setting. We only require the tasks to be described by 
closed trajectories in state spaces, where the states capture the 
relevant movements. Based on oscillator theory, the closed state 
trajectories are interpreted as limit cycles, for which corresponding 
phase variables are derived. The sought phases reflect the expected 
non-stationarity in the limit cycles instantaneously, or in other 
words, they are defined on a within-period scope and determined 
on-line. Goal-directed repetitive movements are shown to contain 
much richer information concerning synchronization than purely 
their oscillating property captured by the phase variable. Through 
segmentation, we split complex movement trajectories into 
sequences of multiple primitives, which are separated by events, 
e.g. the occurrence of points with zero velocity. Beyond in-phase 
and anti-phase known from harmonic oscillations, enhanced 
synchronization modes within limit cycle pairs are synthesized. 
Their definition utilizes both continuous phases and discrete 
events as anchoring points for synchronization. The key idea of the 
synchronization concept is the design of interactive behavior 
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